System and method for prioritizing and balancing simultaneous audio outputs in a handheld device

ABSTRACT

A system and method for prioritizing multiple audio sources and balancing them in a single combined output in a handheld device. Audio sources are sensed and combined with the level of each source subject to adjustment by an attenuator/amplifier. A priority logic unit controls the attenuator/amplifier associated with each source. the level of each source being combined is adjusted on the basis of the character of the sources being combined and a set of programmed rules. For example, if source A is a high priority source (e.g. a telephone ring or other alert tone) and source B is a lower priority source (e.g. a music program) then the sound management system may lower the volume on source B combine with source A and output the result. Alternatively, source A may be raised in volume, combined with source B, and then output. In one embodiment, the sound management system is integrated with a palm sized handheld computer system.

CROSS REFERENCE TO RELATED APPLICATIONS

This application is a continuation application of application Ser. No.09/871,465, entitled “SYSTEM AND METHOD FOR PRIORITIZING AND BALANCINGSIMULTANEOUS AUDIO OUTPUTS IN A HANDHELD DEVICE”, filed May 30, 2001, tobe issued as U.S. Pat. No. 7,272,232. The subject matter in theabove-identified co-pending and commonly owned application(s) isincorporated herein by reference.

BACKGROUND OF THE INVENTION

1. Field of the Invention

This invention relates to audio outputs in handheld devices. Inparticular, the invention relates to the handling of simultaneous audiooutputs.

2. Prior Art

Over the years, many electronic devices have been sufficiently reducedin size to become portable, and in turn, small enough to be held andoperated in the hand of the user. The advent of integrated circuitsgreatly increased the number of devices that could achieve portabilityand handheld status, as well as increasing the functionality of suchdevices.

Initially, handheld devices such as pagers, telephones and radios weresingle purpose devices that had a single audio source and a single audiooutput. For such devices, there was no contention or confusion betweenmultiple audio sources.

As handheld devices became more sophisticated, they began to havemultiple audio sources. Audio sources can be divided into two classes:signal events and continuous sources. In previous devices with multipleaudio sources, the sources were usually either signal events (e.g.ringers or alarms), or continuous sources (e.g. speech or music), butnot both. The audio output of such devices with multiple sources wasobtained by prioritizing the inputs and selecting among them, or notprioritizing the inputs and simply combining them.

For purposes of this disclosure, prioritizing of audio sources isdefined as establishing a ranking of relative importance between audiosources. Balancing of an audio output is defined as establishing apredetermined relative level between at least two audio sources that arepresent in an audio output.

Although telephones utilize both transient and continuous sources, theyare typically mutually exclusive in use. For a service such as callwaiting, a signal event may be introduced to the audio output of thereceiving party's device by superimposing it on the conversation thatprompted the call waiting signal, or by blanking the conversation. Thisis an example of simple mixing of two sources. Simple mixing istolerable in telephones since the signal event duration is short and anyloss of conversation can usually be recovered by an inquiry to the otherparty in the conversation that was interrupted. However, when a signalevent is simply mixed with a source such as a radio broadcast,information lost through the interruption is usually not recoverable.

As the functional capabilities of handheld devices has increased, thenumber of audio sources that may be incorporated in a single device hasexpanded. The development of low cost memory and efficient compressionalgorithms has lead to a new generation of devices that are capable ofproviding both entertainment and communications functions. An example ofsuch a device is a telephone that is capable of playing MP3 files.

The number of available audio sources in handheld devices has increasedthe probability that two sources may interfere. For example, a telephonehaving a stored music playback capability may ring while the music isplaying. Typically, in the case of simultaneous signal event andcontinuous source combinations, the solution has been either to mix thesources or temporarily suppress the continuous source. Mixing of the twosources can render the information contained in one or both sourcesunintelligible. For example, if a music program is too loud, a signalevent such as a phone ring or low battery alarm may not be heard. Thesuppression of one source results in the information from that sourcebeing lost during the period of suppression.

The combination of two continuous sources in a handheld device isusually dealt with by selecting one or the other, as in a radio/tapeplayer. This is an example of prioritizing the inputs and producing asingle output. However, there are situations when it would be desirableto combine two continuous sources and prioritize them with respect tothe combined output.

For example, a device combining a wireless baby monitor and an FM bandradio would have two continuous audio sources. The baby monitor signalcould be received at the 2.4 GHz ISM band, and the radio signal at thecommercial FM band. As a normal condition, the two sources would becombined so that the net effect is that of being in a single room with ababy and a radio. However, it would be desirable to intelligentlycontrol the relative sound levels to guarantee that the baby could beheard when crying, regardless of how loud the radio program was playing.

SUMMARY OF THE INVENTION

Thus, a need exists for a method of prioritizing and balancingsimultaneous audio outputs in a handheld device while minimizing overallinformation loss. There is also a need for a device that can prioritizemultiple continuous audio sources in addition to prioritizing acontinuous audio source and a signal event audio source.

Accordingly, it is an object of the present invention to provide amethod of prioritizing two or more audio sources and balancing thecombined output to minimize information loss. It is a further object ofthe present invention to provide a capability to monitor and prioritizetwo continuous audio sources and balance them in a combined output. Itis also an object to provide flexibility in programming theprioritization and balancing. These and other objects and advantages ofthe present invention and others not specifically recited above will bedescribed in more detail herein.

A system and method for prioritizing multiple audio sources andbalancing them in a single combined output in a handheld device aredisclosed. Audio sources are sensed and combined with the level of eachsource subject to adjustment by an attenuator/amplifier. A prioritylogic unit controls the attenuator/amplifier associated with eachsource. The level of each source being combined is adjusted on the basisof the character of the sources being combined and a set of programmedrules for prioritization.

In one embodiment of the present invention, two audio sources A and Bare sensed by a priority logic unit. Source A is a continuous audiosource and source B is either a continuous audio source or a signalevent audio source. The two sources are combined into a single outputwith each source having a predetermined level of attenuation or gain andthus a predetermined signal level ratio. Upon sensing an increase inamplitude of source B above a preset threshold level, the attenuation orgain of one or both sources is adjusted such that a new signal ratio isestablished between the two sources.

For example, if source A is a high priority source (e.g. a telephonering or other alert tone) and source B is a lower priority source (e.g.a music program) then the sound management system may lower the volumeon source B, combine with source A and output the result. Alternatively,source A may be raised in volume, combined with source B, and thenoutput. In one embodiment, the sound management system is integratedwith a palm sized handheld computer system.

BRIEF DESCRIPTION OF THE DRAWINGS

The accompanying drawings, which are incorporated in and form a part ofthis specification, illustrate embodiments of the invention and,together with the description, serve to explain the principles of theinvention:

FIG. 1A illustrates a representative handheld device.

FIG. 1B illustrates exemplary circuitry of a portable computer system.

FIG. 2 shows a functional block diagram for a system in accordance withan embodiment of the present claimed invention.

FIG. 3 shows a method of prioritizing a signal event audio source over acontinuous audio source in accordance with an embodiment of the presentclaimed invention.

FIG. 4 shows a method of prioritizing a signal event audio source and acontinuous audio source in accordance with an embodiment of the presentclaimed invention.

FIG. 5 shows a method of prioritizing two continuous audio sources inaccordance with an embodiment of the present claimed invention.

DETAILED DESCRIPTION OF THE INVENTION

In the following detailed description of the present invention, a systemand method for prioritizing and balancing a combined audio output,numerous specific details are set forth in order to provide a thoroughunderstanding of the present invention. However, it will be obvious toone skilled in the art that the present invention may be practicedwithout these specific details. In other instances well known methods,procedures, components, and circuits have not been described in detailso as not to unnecessarily obscure aspects of the present invention

FIG. 1A shows a representative handheld device 100 that is soundenabled. Audio sources that may be incorporated into such a handhelddevice include radio frequency and infrared reception of analog anddigital signals. These signals may be commercial broadcast, telephonecalls or data transmitted from a wireless network. BLUETOOTH is aparticular short range wireless technology that can link a telephone, aradio, or other device to a digital data source.

In addition to received signals, the handheld device 100 of FIG. 1A mayalso be capable of generating signal events such as an alarm associatedwith a clock, timer, thermometer, or phone battery monitor. The devicemay also have internal audio sources such as speech or music stored indigital or analog form. Fixed or removable media may include magnetic,optical, and integrated circuit storage media. The magnetic media may beused for analog or digital storage. Alternatively, the speech or musicmaterial may be stored on a digital storage medium such as flash memoryor random access memory (RAM). Speech or music may be stored in avariety of formats such as MP3 for music or Adaptive Differential PulseCode Modulation (ADPCM) for speech.

FIG. 1B illustrates exemplary schematic circuitry of portable computersystem 100. Computer system 100 includes an address/data bus 99 forcommunicating information. It should be understood that the general bus99 may consist of a number of buses with different data path widths andoperating frequencies, coupled by bridge circuits. The computer system100 may or may not contain all of the functional blocks shown. Computersystem 100 includes a central processor 101 coupled with the bus 99 forprocessing information and instructions, a volatile memory 102 (e.g.,random access memory RAM) coupled with the bus 99 for storing staticinformation and instructions for the central processor 101 and anon-volatile memory 103 (e.g., read only memory ROM) coupled with thebus 99 for storing static information and instructions for the processor101. Computer system 100 also includes an optional data storage device104 (e.g., thin profile removable memory) coupled with the bus 99 forstoring information and instructions. Device 104 can be removable, suchas a flash memory card. As described above, system 100 also contains adisplay device 105 coupled to the bus 99 for displaying information tothe computer user.

Also included in computer system 100 of FIG. 1B is an alphanumeric inputdevice 106. Device 106 can communicate information (spatial data andpressure data) and command selections to the central processor 101.

System 100 also includes an optional cursor control or directing device107 coupled to the bus for communicating user input information andcommand selections to the central processor 101. In one implementation,device 107 is a touch screen device (also a digitizer) incorporated withscreen 105. Device 107 is capable of registering a position on thescreen 105 where the stylus makes contact and the pressure of thecontact. The digitizer can be implemented using well known devices, forinstance, using the ADS-7846 device by Burr-Brown that provides separatechannels for spatial stroke information and pressure information.

The display device 105 utilized with the computer system 100 may be aliquid crystal display (LCD) device, cathode ray tube (CRT), fieldemission display (FED, also called flat panel CRT) or other displaydevice suitable for creating graphic images and alphanumeric charactersrecognizable to the user. Any of a number of display technologies can beused, e.g., LCD, FED, plasma, etc., for the flat panel display 105. Inone embodiment, the display 105 is a flat panel multi-mode displaycapable of both, monochrome and color display modes.

Signal communication device 108, also coupled to bus 99, can be a serialport or USB port. In addition to device 108, wireless communicationlinks can be established between device 100 and a host computer system(or another portable computer system) using a BLUETOOTH wireless device111, an infrared device 110. Device 100 may also include a wirelessreceiver device 109.

In one implementation, the Mobitex wireless communication system may beused to provide two way communication between system 100 and othernetworked computers and/or the Internet via a proxy server. In otherembodiments, TCP protocol can be used or SMS (Short Message Service) canbe used. System 100 of FIG. 1B may also contain batteries for providingelectrical power. Replaceable cells or rechargeable batteries can beused. Well known electronics may be coupled to the battery to detect itsenergy level and this information can be sampled by the processor 101.

Several of the devices shown in FIG. 1B may serve as audio sources.Audio program material may be stored in RAM 120, ROM 103, and DataStorage Device 104. Additionally, serial port 108, the wireless receiver109, the infrared device 110, the BLUETOOTH device 111 are capable ofreceiving audio program material in analog or digital formats. Adedicated audio source 112 may be an audio synthesizer or an input for amicrophone or other audio input. Also, the processor 101 may also serveas a digital audio synthesizer in conjunction with appropriate software.

FIG. 2 shows a system embodiment of the invention involving two sources,audio source A 200 and audio source B 201. Although the principles ofthe invention are applicable to systems having more than two sources,the embodiment shown provides two sources in order to describe thefundamental operation of the invention.

Audio source A 200 and audio source B 201 may be either a signal eventsource or a continuous source. A signal event source is an audio signalof short duration such as a telephone ring or an alarm. A signal eventis an audio signal that is intended to alert a user to a change in stateor condition and is triggered by the change in state or condition. Acontinuous source is an audio source with a typically longer durationthan a signal event source, and is usually not dependent upon anexternal event, but is selected arbitrarily by the user. Audio source A200 is coupled to a variable attenuator/amplifier 203 and audio source B201 is coupled to a variable attenuator/amplifier 204.

The variable attenuator/amplifier is typically used to provide gain to alow level signal and thus act as an amplifier. In some cases, where ahigh level input is used, it may act as an attenuator. The attenuationand amplification may be done by analog or digital techniques and maycombine analog to digital conversion or digital to analog conversion.The variable attenuator/amplifier may be implemented entirely insoftware.

Referring again to FIG. 2, a priority logic unit 202 is coupled to audiosource A 200 and audio source B 201. At a minimum, the priority logicunit 202 is capable of sensing the amplitude and/or presence of eachaudio source. Optionally, the priority logic unit 202 may have a memorybuffer that is capable of storing an interval of the signal beingsensed. The priority logic unit 202 is coupled to variableattenuator/amplifier 203 and is also coupled to variableattenuator/amplifier 204. The priority logic unit 202 is furnished witha set of prioritization rules allowing it to control the amount of gainor attenuation that is applied to audio source A 200 by variableattenuator/amplifier 203, and the amount of gain or attenuation that isapplied to audio source B 201 by variable attenuator/amplifier 204.

There are many possible prioritization rules that can be used toprioritize the two audio sources. As an example, Audio source A 200 maybe attenuated to the point of being effectively muted whenever Audiosource B 201 is active (i.e. has a non-zero amplitude).

For each possible audio source pair in the handheld device, theprioritization rules establish the relative gain applied to the sources.the absolute levels may be set in relation to a fixed decibel level, orit may be referenced to the level or presence of one of the sources.Alternatively, prioritization between a signal event and a continuoussource may be dependent upon the presence of a continuous source,regardless of the immediate sound level. For instance a .WAV file or.MP3 file that is being played. The prioritization rules establish ahierarchy of priority for the audio sources in the handheld device. Therules may be implemented in hardware or software, or both.

The output of variable attenuator/amplifier 203 and variableattenuator/amplifier 204 are coupled to a mixer 205 that combines thetwo signals into a single output 206. The output 206 may be coupled to aspeaker, headphone jack, or line out. The output 206 may consist of asingle channel of a stereophonic pair, in which case, a handheld devicemay have more than one implementation of the system shown in FIG. 2, asin two channels forming a stereophonic pair.

It should be noted that signal path A 207 and signal path B 208 maycarry an analog signal, digital signal, or a piecewise combination ofboth, with analog/digital (A/D) or digital/analog (D/A) conversion beingapplied at any point in the signal paths between audio source A 200 andaudio source B 201, and the output 206. The priority logic unit may be adigital device, but is capable of performing A/D and D/A conversion asrequired in order to provide source sensing and control of the variableattenuator/amplifiers.

For example, if source A is a high priority source (e.g. a telephonering or other alert tone) and source B is a lower priority source (e.g.a music program) then the sound management system may lower the volumeon source B, combine with source A and output the result. Alternatively,source A may be raised in volume, combined with source B, and thenoutput. In one embodiment, the sound management system is integratedwith a palm sized handheld computer system.

FIG. 3 shows a particular embodiment of the invention wherein audiosource B is a signal event source that has a higher priority than acontinuous audio source A. At time T₀ Mixer input A has an arbitraryinitial level A_(I), and audio source B is inactive with Mixer input Bhaving a value of zero. Mixer input A and Mixer input B are the signalsderived from audio source A 200 and audio source B of FIG. 1, and havebeen processes by the variable attenuator/amplifiers in concert with thepriority logic unit 202. In this scenario, Input A could be derived froma continuous source such as a radio broadcast, and Input B could bederived from a signal event input such as a telephone ringer. At timeT₁, audio source B becomes active and Mixer input B has a level of B. Inresponse to audio source B becoming active, the system causes Mixerinput B to be reduced to an attenuated level A_(A) A time T₂ audiosource B becomes inactive and Mixer input A is restored to it previouslevel A_(I).

The prioritization scenario of FIG. 3 is particularly useful when theuser of the handheld device has the volume of the radio broadcast setrelatively high, and there is a likelihood that the telephone ring maybe drowned out. At lower listening levels, the telephone ring may becombined with the radio program with adjusting the level of the radioprogram. In this case the prioritization rule applied calls for aparticular gain ratio to make sure that the ring is audible over themusic, and if the ring cannot be sufficiently amplified the radioprogram volume is reduced. In a variation of this rule, the radioprogram volume may always be reduced if the user is using a headphoneoutput in order to prevent possible discomfort from too muchamplification of the telephone ring. A variation in the prioritizationrules such as this may be programmable in that the priority logic unitdetects the nature of the output being used (e.g. headphone or speaker)and the source selection (e.g. tape, radio or microphone) and adjuststhe prioritization rules accordingly. The user may have the opportunityto establish prioritization rules by means of an urgent or non-urgentsetting for the telephone ring.

FIG. 4 shows the effects of a non-urgent setting for a telephone ring.The timing shown in FIG. 4 is the same as that shown in FIG. 3; however,since mixer input A (radio program) now has a higher priority, the levelis unchanged when the audio source B (telephone ring) becomes active,and the ring level is set at a level below the radio program, based uponthe gain ratio associated with the prioritization rule, and referencedto the radio program level.

FIG. 5 shows a scenario in which both sources are continuous sources. Inthis particular embodiment, mixer input A is derived from a radioprogram and audio source B is a wireless baby monitor. Both sources areinitially of equal priority, and they are combined to produce theoutput. At T₀ mixer input A is at an arbitrary initial level A_(I).Between T₀ and T₁ the baby's room is quiet and the mixer output consistsessentially of mixer input A at the initial level A_(I) and the silencein the baby's room. Between T₁ and T₂, there is a slight noise in thebaby's room that is below the threshold value B_(T). Since the noise isbelow the threshold, the output continues to consist of the combinationof radio program at it's initial level and the sounds from the baby'sroom. The period between T₂ and T₃ the situation is identical to thatbetween T₀ and T₁. However, between T₃ and T₄, the noise in the baby'sroom is above the threshold value B_(T). In response to audio source Bcrossing the threshold, the system brings the noise in the baby's roomto the foreground by reducing the level of mixer input A to a levelA_(A) for the period during which the noise in the baby's room is abovethe threshold. In the limiting case, the level A_(A) would be zero.

The previously discussed scenarios have dealt with the combination oftwo continuous audio sources and the combination of a continuous audiosource and signal event source. The remaining combination of two signalevent sources is the least probable situation due to the short durationof signal events, but requires a different approach for balancing.Signal events are typically a single pulse at a given tone, a sequenceof pulses at a single tone, or a sequence of pulses of different tones.If a situation arises where two signal events overlap, the second eventcan be stored by a buffer in the priority logic unit at the same time itis being suppressed by its associated variable attenuator/amplifier.After the first signal event has ended, the priority logic unit canreplay the stored signal event so that both signal events may be heardand overlap is avoided.

It can be seen from the present embodiments that multiple audio sourcesin a handheld device can be prioritized and a balanced output obtainedbased upon the prioritization. The capability for balancing combinedaudio sources in a single output offers more flexibility than simplychoosing between multiple sources or combining them withoutprioritization.

The foregoing descriptions of specific embodiments of the presentinvention have been presented for purposes of illustration anddescription. They are not intended to be exhaustive or to limit theinvention to the precise forms disclosed, and obviously manymodifications and variations are possible in light of the aboveteaching. The embodiments were chosen and described in order to bestexplain the principles of the invention and its practical application,to thereby enable others skilled in the art to best utilize theinvention and various embodiments with various modifications are suitedto the particular use contemplated. It is intended that the scope of theinvention be defined by the claims appended hereto and theirequivalents.

1. A handheld device comprising: a) a first audio source coupled to afirst variable attenuator/amplifier; b) a second audio source coupled toa second variable attenuator/amplifier; c) a priority logic unit forassigning priority levels to coupled to said first audio source and saidsecond audio source, and also coupled to said first variableattenuator/amplifier and said second variable attenuator/amplifier; d) amixer coupled to said first variable attenuator/amplifier and to saidsecond variable attenuator/amplifier; and e) an audio output connectedto said mixer.